packages/sample-app/sample_app/experiment/made_by_traceloop/travel_agent_exp.py (2)

31-38: Guard dynamic travel agent loading for clearer errors

_load_travel_agent_module assumes the agents_path exists and that spec/spec.loader is non-None. If the file is missing or the path changes, this will fail with a somewhat opaque AttributeError.

You could make failures clearer:

 def _load_travel_agent_module():
     """Dynamically load the travel_agent_example module."""
     agents_path = Path(__file__).parent.parent.parent / "agents" / "travel_agent_example.py"
-    spec = importlib.util.spec_from_file_location("travel_agent_example", agents_path)
-    module = importlib.util.module_from_spec(spec)
-    sys.modules["travel_agent_example"] = module
-    spec.loader.exec_module(module)
-    return module
+    if not agents_path.exists():
+        raise FileNotFoundError(f"travel_agent_example.py not found at {agents_path}")
+
+    spec = importlib.util.spec_from_file_location("travel_agent_example", agents_path)
+    if spec is None or spec.loader is None:
+        raise ImportError(f"Could not load spec for travel_agent_example from {agents_path}")
+
+    module = importlib.util.module_from_spec(spec)
+    sys.modules["travel_agent_example"] = module
+    spec.loader.exec_module(module)
+    return module

This keeps the example robust while failing fast with a helpful message if the file isn’t where it’s expected.

---

185-219: Normalize trajectory JSON handling and include tool_calls in the task result

The current trajectory handling has a few rough edges:

• For empty trajectories, you set trajectory_prompts = json.dumps([]) / trajectory_completions = json.dumps([]) and then later call json.dumps again, which yields "[]" (a JSON string of a JSON string) instead of [].
• The “Prompt attributes captured” / “Completion attributes captured” counts are based on mixed dict/string types, so counts become misleading when empty.
• The returned task result doesn’t expose tool_calls, even though they’re collected and later referenced in the sample results.

You can address all three by:

-        # trajectory_prompts and trajectory_completions are dicts with llm.prompts/completions.* keys
-        # If empty, use JSON string fallback to avoid validation errors
-        trajectory_prompts = trajectory_data["trajectory_prompts"]
-        trajectory_completions = trajectory_data["trajectory_completions"]
-
-        # Convert to JSON strings if empty (evaluators expect string when no data)
-        if not trajectory_prompts:
-            trajectory_prompts = json.dumps([])
-        if not trajectory_completions:
-            trajectory_completions = json.dumps([])
-
-        print("📊 Trajectory Summary:")
-        print(f"  - Prompt attributes captured: {len(trajectory_prompts)}")
-        print(f"  - Completion attributes captured: {len(trajectory_completions)}")
+        # trajectory_prompts and trajectory_completions are dicts with llm.prompts/completions.* keys
+        trajectory_prompts = trajectory_data["trajectory_prompts"]
+        trajectory_completions = trajectory_data["trajectory_completions"]
+
+        prompt_attr_count = len(trajectory_prompts)
+        completion_attr_count = len(trajectory_completions)
+
+        # Convert to JSON strings for evaluator input; use [] when empty
+        json_trajectory_prompts = json.dumps(trajectory_prompts or [])
+        json_trajectory_completions = json.dumps(trajectory_completions or [])
+
+        print("📊 Trajectory Summary:")
+        print(f"  - Prompt attributes captured: {prompt_attr_count}")
+        print(f"  - Completion attributes captured: {completion_attr_count}")
@@
-        # - "context" maps to "reference"
-        # - "trajectory_prompts" and "trajectory_completions" as dicts with llm.prompts/completions.* keys
-        
-        json_trajectory_prompts = json.dumps(trajectory_prompts)
-        json_trajectory_completions = json.dumps(trajectory_completions)
-        # prompt_list = str(trajectory_prompts)
-        # completion_list = str(trajectory_completions)
-
-        return {
-            "prompt": query,
-            "answer": final_completion if final_completion else query,
-            "context": f"The agent should create a complete travel itinerary for: {query}",
-            "trajectory_prompts": json_trajectory_prompts,  
-            "trajectory_completions": json_trajectory_completions,  
-        }
+        # - "context" maps to "reference"
+        # - "trajectory_prompts" and "trajectory_completions" as JSON strings with llm.prompts/completions.* keys
+
+        return {
+            "prompt": query,
+            "answer": final_completion if final_completion else query,
+            "context": f"The agent should create a complete travel itinerary for: {query}",
+            "trajectory_prompts": json_trajectory_prompts,
+            "trajectory_completions": json_trajectory_completions,
+            "tool_calls": trajectory_data["tool_calls"],
+        }

This keeps evaluator inputs JSON-encoded (with a clean [] for the empty case), fixes the counts, and surfaces tool_calls for downstream inspection.

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

Learnt from: CR
Repo: traceloop/openllmetry PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-08-17T15:06:48.109Z
Learning: For debugging OpenTelemetry spans, use ConsoleSpanExporter with Traceloop to print spans to console

packages/sample-app/sample_app/experiment/made_by_traceloop/travel_agent_exp.py (4)

105-111: Ensure returned task fields match documented “Required fields” and planned evaluators

Your docstring lists tool_calls as a required field, and you’re already computing tool calls (plus tool inputs/outputs) in extract_trajectory_from_spans, but the task result currently omits them. The diff above adds tool_calls; if you intend to plug in agent_tool_error_detector later, consider also returning tool_inputs and tool_outputs from trajectory_data so the evaluator has everything it needs.

Also applies to: 182-197

---

269-278: Result summary uses wrong keys and doesn’t decode JSON trajectories

In the sample results, you read prompts/completions, but travel_agent_task returns trajectory_prompts/trajectory_completions as JSON strings. That will always fall back to '' and counts won’t reflect the captured data. You can decode the JSON and count elements instead:

-                task_result = result.task_result
-                print(f"    - Query: {task_result.get('prompt', 'N/A')[:100]}...")
-                print(f"    - Tools used: {len(task_result.get('tool_calls', []))}")
-                print(f"    - Prompts captured: {len(task_result.get('prompts', '').split('---'))}")
-                print(f"    - Completions captured: {len(task_result.get('completions', '').split('---'))}")
+                task_result = result.task_result
+                print(f"    - Query: {task_result.get('prompt', 'N/A')[:100]}...")
+                print(f"    - Tools used: {len(task_result.get('tool_calls', []))}")
+                prompts_str = task_result.get("trajectory_prompts", "[]")
+                completions_str = task_result.get("trajectory_completions", "[]")
+                try:
+                    prompts_data = json.loads(prompts_str)
+                except (TypeError, json.JSONDecodeError):
+                    prompts_data = []
+                try:
+                    completions_data = json.loads(completions_str)
+                except (TypeError, json.JSONDecodeError):
+                    completions_data = []
+                print(f"    - Prompts captured: {len(prompts_data)}")
+                print(f"    - Completions captured: {len(completions_data)}")

This keeps the summary aligned with the actual task output format.

---

1-10: Evaluator list is inconsistent (docs/prints say 5, code runs 4)

Module and function docstrings plus console output describe five evaluators including “Agent Tool Error Detector”, but evaluators only contains four (agent_goal_accuracy, agent_flow_quality, agent_efficiency, agent_goal_completeness). Either add EvaluatorMadeByTraceloop.agent_tool_error_detector(...) (and wire the required fields into travel_agent_task) or update all three locations to describe only the four actually used, to avoid confusing users of the example.

Also applies to: 203-224

---

129-197: Clean up trajectory encoding, answer fallback, and outer try/except in travel_agent_task

This block has a few issues: double‑JSON‑encoding empty trajectories, using the query text as a fallback “answer”, and a try/except that just re‑raises the same exception (flagged by Ruff). The example below normalizes empties without double‑encoding, keeps attribute counts correct, returns a safer fallback answer, includes tool_calls in the task result, and removes the useless try/except:

-    try:
-        # Run the travel agent query
-        print(f"\n{'='*80}")
-        print(f"Running travel agent for query: {query}")
-        print(f"{'='*80}\n")
-
-        tool_calls_made = await run_travel_query(query)
-
-        # Get all captured spans
-        spans = exporter.get_finished_spans()
-
-        print(f"\n{'='*80}")
-        print(f"Captured {len(spans)} spans from travel agent execution")
-        print(f"{'='*80}\n")
-
-        # Extract trajectory from spans
-        trajectory_data = extract_trajectory_from_spans(spans)
-
-        # Get the final completion from llm.completions dict
-        completions_dict = trajectory_data["trajectory_completions"]
-        final_completion = ""
-        if completions_dict:
-            # Find the highest index completion content
-            max_idx = -1
-            for key in completions_dict.keys():
-                if ".content" in key:
-                    try:
-                        parts = key.split(".")
-                        idx = int(parts[2])
-                        if idx > max_idx:
-                            max_idx = idx
-                            final_completion = completions_dict[key]
-                    except (ValueError, IndexError):
-                        pass
-
-        # trajectory_prompts and trajectory_completions are dicts with llm.prompts/completions.* keys
-        # If empty, use JSON string fallback to avoid validation errors
-        trajectory_prompts = trajectory_data["trajectory_prompts"]
-        trajectory_completions = trajectory_data["trajectory_completions"]
-
-        # Convert to JSON strings if empty (evaluators expect string when no data)
-        if not trajectory_prompts:
-            trajectory_prompts = json.dumps([])
-        if not trajectory_completions:
-            trajectory_completions = json.dumps([])
-
-        print("📊 Trajectory Summary:")
-        print(f"  - Prompt attributes captured: {len(trajectory_prompts)}")
-        print(f"  - Completion attributes captured: {len(trajectory_completions)}")
-        tools_called = ', '.join(trajectory_data['tool_calls']) if trajectory_data['tool_calls'] else 'None'
-        print(f"  - Tools called: {tools_called}")
-        print(f"  - Tools from run: {', '.join(tool_calls_made) if tool_calls_made else 'None'}\n")
-
-        # Return data using field synonyms that map to required evaluator fields
-        # - "prompt" maps to "question"
-        # - "answer" maps to "completion"
-        # - "context" maps to "reference"
-        # - "trajectory_prompts" and "trajectory_completions" as JSON strings
-
-        json_trajectory_prompts = json.dumps(trajectory_prompts)
-        json_trajectory_completions = json.dumps(trajectory_completions)
-
-        return {
-            "prompt": query,
-            "answer": final_completion if final_completion else query,
-            "context": f"The agent should create a complete travel itinerary for: {query}",
-            "trajectory_prompts": json_trajectory_prompts,
-            "trajectory_completions": json_trajectory_completions,
-        }
-
-    except Exception as e:
-        raise e
+    # Run the travel agent query
+    print(f"\n{'='*80}")
+    print(f"Running travel agent for query: {query}")
+    print(f"{'='*80}\n")
+
+    tool_calls_made = await run_travel_query(query)
+
+    # Get all captured spans
+    spans = exporter.get_finished_spans()
+
+    print(f"\n{'='*80}")
+    print(f"Captured {len(spans)} spans from travel agent execution")
+    print(f"{'='*80}\n")
+
+    # Extract trajectory from spans
+    trajectory_data = extract_trajectory_from_spans(spans)
+
+    # Get the final completion from llm.completions dict
+    completions_dict = trajectory_data["trajectory_completions"]
+    final_completion = ""
+    if completions_dict:
+        # Find the highest index completion content
+        max_idx = -1
+        for key in completions_dict.keys():
+            if ".content" in key:
+                try:
+                    parts = key.split(".")
+                    idx = int(parts[2])
+                    if idx > max_idx:
+                        max_idx = idx
+                        final_completion = completions_dict[key]
+                except (ValueError, IndexError):
+                    pass
+
+    # trajectory_prompts and trajectory_completions are dicts with llm.prompts/completions.* keys
+    trajectory_prompts = trajectory_data["trajectory_prompts"]
+    trajectory_completions = trajectory_data["trajectory_completions"]
+
+    prompt_attr_count = len(trajectory_prompts)
+    completion_attr_count = len(trajectory_completions)
+
+    # Normalise empty trajectories so we don't double‑encode JSON
+    if not trajectory_prompts:
+        trajectory_prompts = []
+    if not trajectory_completions:
+        trajectory_completions = []
+
+    print("📊 Trajectory Summary:")
+    print(f"  - Prompt attributes captured: {prompt_attr_count}")
+    print(f"  - Completion attributes captured: {completion_attr_count}")
+    tools_called = ', '.join(trajectory_data['tool_calls']) if trajectory_data['tool_calls'] else 'None'
+    print(f"  - Tools called: {tools_called}")
+    print(f"  - Tools from run: {', '.join(tool_calls_made) if tool_calls_made else 'None'}\n")
+
+    # Return data using field synonyms that map to required evaluator fields
+    # - "prompt" maps to "question"
+    # - "answer" maps to "completion"
+    # - "context" maps to "reference"
+    # - "trajectory_prompts" and "trajectory_completions" as JSON strings
+
+    json_trajectory_prompts = json.dumps(trajectory_prompts)
+    json_trajectory_completions = json.dumps(trajectory_completions)
+
+    return {
+        "prompt": query,
+        "answer": final_completion if final_completion else "No completion captured",
+        "context": f"The agent should create a complete travel itinerary for: {query}",
+        "trajectory_prompts": json_trajectory_prompts,
+        "trajectory_completions": json_trajectory_completions,
+        "tool_calls": trajectory_data.get("tool_calls", []),
+    }

This should also clear the Ruff TRY201/203/300 and the earlier comments about double‑encoding and the semantically odd answer fallback.

Also applies to: 199-200

packages/sample-app/sample_app/experiment/made_by_traceloop/travel_agent_exp.py (2)

26-31: Unused # noqa: E402 directive on run_travel_query import

Ruff reports this noqa as unused; if E402 is not enabled in your lint config, you can drop the directive to reduce noise. If you do rely on Flake8’s E402, consider a dynamic import (e.g., via importlib) instead, so the import can live at the top of the file without needing # noqa.

---

37-51: extract_trajectory_from_spans docstring doesn’t match behavior

The docstring says you convert gen_ai.prompt.* / gen_ai.completion.* to llm.prompts.* / llm.completions.*, but the implementation simply filters gen_ai.* keys without renaming. Please either (a) actually remap the keys to the llm.* prefix, or (b) update the docstring/inline comment so expectations for downstream evaluators are accurate.

Also applies to: 61-66

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

packages/sample-app/sample_app/experiment/made_by_traceloop/travel_agent_exp.py (7)

1-14: Docstring lists 5 evaluators but only 4 are instantiated.

The module docstring references "Agent Tool Error Detector" at line 7, but the evaluators list at lines 228-236 only instantiates four evaluators (missing agent_tool_error_detector). This has been flagged in previous reviews.

---

193-193: Fallback to query as answer is semantically incorrect.

If final_completion is empty, returning the input query as the answer conflates the question with the response. This has been flagged in previous reviews.

---

199-200: Remove unnecessary try/except block.

The try/except at lines 129-200 catches and immediately re-raises without adding logging or context. This has been flagged in previous reviews.

---

219-224: Printed list claims 5 evaluators but only 4 are instantiated.

The console output lists "Agent Tool Error Detector" as evaluator #2, but it's not instantiated in the evaluators list at lines 228-236. This has been flagged in previous reviews.

---

277-278: Field name mismatch in result inspection.

Lines 277-278 access task_result.get('prompts', '') and task_result.get('completions', ''), but travel_agent_task returns keys named trajectory_prompts and trajectory_completions (lines 195-196). This has been flagged in previous reviews.

---

166-173: Avoid double JSON encoding for empty trajectories.

When trajectory_prompts or trajectory_completions are empty dicts, lines 171 and 173 convert them to JSON strings (json.dumps([])). Then lines 188-189 call json.dumps() again on these strings, resulting in double-encoded JSON strings like "\"[]\"" instead of "[]".

Apply this diff to initialize with empty lists instead:

-        # Convert to JSON strings if empty (evaluators expect string when no data)
+        # Initialize empty dicts as empty lists for consistent JSON encoding
         if not trajectory_prompts:
-            trajectory_prompts = json.dumps([])
+            trajectory_prompts = []
         if not trajectory_completions:
-            trajectory_completions = json.dumps([])
+            trajectory_completions = []

---

37-47: Docstring incorrectly claims format conversion.

The docstring at line 40 states "Converts gen_ai.prompt.* to llm.prompts.* format expected by evaluators" and line 46 claims the return dict has "llm.prompts.* keys", but the code at line 63 stores keys unchanged (still as gen_ai.prompt.*). No conversion occurs.

Apply this diff to correct the docstring:

 def extract_trajectory_from_spans(spans):
     """
     Extract prompt and completion trajectory from OpenTelemetry spans.
-    Converts gen_ai.prompt.* to llm.prompts.* format expected by evaluators.
 
     Args:
         spans: List of ReadableSpan objects from InMemorySpanExporter
 
     Returns:
-        dict with trajectory_prompts (as dict with llm.prompts.* keys), trajectory_completions, and tool_calls
+        dict with trajectory_prompts (as dict with gen_ai.prompt.* keys), trajectory_completions (as dict with gen_ai.completion.* keys), and tool_calls
     """

packages/sample-app/sample_app/experiment/made_by_traceloop/travel_agent_exp.py (1)

31-31: Remove unused noqa directive.

The noqa: E402 comment is unnecessary as the E402 rule is not enabled in your linting configuration.

Apply this diff:

-from travel_agent_example import run_travel_query  # noqa: E402
+from travel_agent_example import run_travel_query

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

packages/traceloop-sdk/traceloop/sdk/experiment/utils.py (1)

141-160: Fix empty-trajectory handling to avoid double JSON encoding and incorrect counts.

Current flow for empty trajectories:

• trajectory_prompts / trajectory_completions start as {}.
• For empties, they’re set to json.dumps([]) (a string "[]").
• Later both are wrapped again with json.dumps(...), yielding "\"[]\"".
• len(trajectory_prompts) / len(trajectory_completions) then measure string length (2), not attribute counts.

This breaks consumers expecting a single-layer JSON string and makes the summary counts wrong.

Suggested change:

• Keep the internal values as structured objects (dict/list), only JSON-encode once at the end:

-        trajectory_prompts = trajectory_data["trajectory_prompts"]
-        trajectory_completions = trajectory_data["trajectory_completions"]
-
-        # Convert to JSON strings if empty (evaluators expect string when no data)
-        if not trajectory_prompts:
-            trajectory_prompts = json.dumps([])
-        if not trajectory_completions:
-            trajectory_completions = json.dumps([])
+        trajectory_prompts = trajectory_data["trajectory_prompts"]
+        trajectory_completions = trajectory_data["trajectory_completions"]
+
+        # Use an empty list as the canonical “no trajectory” structure
+        if not trajectory_prompts:
+            trajectory_prompts = []
+        if not trajectory_completions:
+            trajectory_completions = []
@@
-        print(f"  - Prompt attributes captured: {len(trajectory_prompts)}")
-        print(f"  - Completion attributes captured: {len(trajectory_completions)}")
+        print(f"  - Prompt attributes captured: {len(trajectory_prompts)}")
+        print(f"  - Completion attributes captured: {len(trajectory_completions)}")
@@
-        json_trajectory_prompts = json.dumps(trajectory_prompts)
-        json_trajectory_completions = json.dumps(trajectory_completions)
+        json_trajectory_prompts = json.dumps(trajectory_prompts)
+        json_trajectory_completions = json.dumps(trajectory_completions)

This preserves summary semantics and returns a single JSON layer while keeping the earlier reviewer feedback about empty lists in mind.

packages/sample-app/sample_app/experiment/made_by_traceloop/travel_agent_exp.py (2)

1-14: Docstrings and console text advertise 5 evaluators, but only 4 are instantiated.

• Module and function docstrings, plus the printed description, list “Agent Tool Error Detector” and say there are five evaluators.
• The actual evaluators list only includes:
  • agent_goal_accuracy
  • agent_flow_quality
  • agent_efficiency
  • agent_goal_completeness

Please either:

• Add EvaluatorMadeByTraceloop.agent_tool_error_detector(...) to the list (and ensure the task output exposes whatever tool_input / tool_output fields that evaluator requires), or
• Update all docstrings and printed descriptions to only mention the four evaluators you actually run.

Right now the example is misleading for users following the documented behavior.

Also applies to: 71-93, 95-104

---

35-68: Avoid using the input query as the fallback “answer”.

In travel_agent_task, the returned payload uses:

"answer": final_completion if final_completion else query,

When no completion is captured, this reports the original question as the answer, which corrupts downstream evaluation (e.g., goal accuracy, flow quality) and makes debugging harder.

Prefer a neutral or explicit fallback, e.g.:

-        "answer": final_completion if final_completion else query,
+        "answer": final_completion if final_completion else "No completion captured",

(or an empty string / None, depending on how the evaluators handle missing completions).

packages/traceloop-sdk/traceloop/sdk/experiment/utils.py (1)

163-164: Remove no-op try/except wrapper around the function body.

The except Exception as e: raise e adds no value and triggers Ruff TRY201/TRY203/TRY300 hints. You can safely drop the outer try/except and let exceptions propagate with their original traceback:

-    try:
-        # Run the task callable
+    # Run the task callable
@@
-        json_trajectory_completions = json.dumps(trajectory_completions)
-
-        return json_trajectory_prompts, json_trajectory_completions, final_completion
-
-    except Exception as e:
-        raise e
+        json_trajectory_completions = json.dumps(trajectory_completions)
+
+        return json_trajectory_prompts, json_trajectory_completions, final_completion

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

packages/sample-app/sample_app/experiment/made_by_traceloop/travel_agent_exp.py (4)

1-14: Docstring lists 5 evaluators but only 4 are implemented.

The module docstring references "Agent Tool Error Detector" on line 7, but the evaluators list at lines 96-104 only instantiates four evaluators (agent_goal_accuracy, agent_flow_quality, agent_efficiency, agent_goal_completeness). Update the docstring to remove line 7 or add the missing evaluator to lines 96-104.

---

62-68: Fallback to query as answer is semantically incorrect.

On line 64, when final_completion is empty, the code returns the input query as the answer. This conflates the question with the response. Use an empty string or a placeholder like "No completion captured" instead.

Apply this diff:

     return {
         "prompt": query,
-        "answer": final_completion if final_completion else query,
+        "answer": final_completion if final_completion else "No completion captured",
         "context": f"The agent should create a complete travel itinerary for: {query}",
         "trajectory_prompts": trajectory_prompts,
         "trajectory_completions": trajectory_completions,
     }

---

73-81: Function docstring lists 5 evaluators but only 4 are implemented.

The docstring references "Agent Tool Error Detector" in item 2, but the evaluators list at lines 96-104 only instantiates four evaluators. Update the docstring to match the actual implementation.

---

88-92: Print statements list 5 evaluators but only 4 are implemented.

Lines 88-92 print a list that includes "Agent Tool Error Detector" on line 89, but the evaluators list at lines 96-104 only instantiates four evaluators. Update the print statements to match the actual implementation or add the missing evaluator.

packages/sample-app/sample_app/experiment/made_by_traceloop/travel_agent_exp.py (1)

29-29: Remove unused noqa directive.

Ruff reports that the noqa: E402 directive is unused. Remove it to clean up the code.

Apply this diff:

-from travel_agent_example import run_travel_query  # noqa: E402
+from travel_agent_example import run_travel_query

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

packages/traceloop-sdk/traceloop/sdk/experiment/utils.py (3)

104-105: Remove the no-op try/except that just re-raises the same exception

The try block covers almost the entire function, and the except Exception as e: raise e handler adds no value (and is what Ruff flags with TRY203/TRY201). Letting exceptions propagate naturally keeps tracebacks intact and simplifies control flow.

You can just drop the try/except entirely:

-    try:
-        # Run the task callable
+    # Run the task callable
@@
-        return json_trajectory_prompts, json_trajectory_completions, final_completion
-
-    except Exception as e:
-        raise e
+        return json_trajectory_prompts, json_trajectory_completions, final_completion

Also applies to: 163-164

---

11-21: Align extract_trajectory_from_spans docstring with actual behavior and returned fields

The docstring still says gen_ai.* attributes are converted to llm.prompts.* / llm.completions.* and that only prompts/completions/tool_calls are returned, but the implementation:

• Keeps keys as gen_ai.prompt.* / gen_ai.completion.* without renaming.
• Also returns tool_inputs and tool_outputs.

Either update the implementation to actually produce llm.prompts.* / llm.completions.* and restrict the return shape, or (probably simpler here) update the docstring to describe that gen_ai.* keys are passed through unchanged and that tool_calls, tool_inputs, and tool_outputs are part of the contract.

Also applies to: 59-65

---

139-160: Fix double JSON-encoding of empty trajectories and incorrect attribute counts

When there are no prompts/completions, trajectory_prompts / trajectory_completions are set to json.dumps([]) and then wrapped again with json.dumps(...), producing "\"[]\"" instead of "[]". It also makes the “Prompt attributes captured” count show len("[]") == 2 instead of 0.

You can avoid both issues and still return empty arrays as JSON like this:

-        # trajectory_prompts and trajectory_completions are dicts with llm.prompts/completions.* keys
-        # If empty, use JSON string fallback to avoid validation errors
-        trajectory_prompts = trajectory_data["trajectory_prompts"]
-        trajectory_completions = trajectory_data["trajectory_completions"]
-
-        # Convert to JSON strings if empty (evaluators expect string when no data)
-        if not trajectory_prompts:
-            trajectory_prompts = json.dumps([])
-        if not trajectory_completions:
-            trajectory_completions = json.dumps([])
+        # trajectory_prompts and trajectory_completions are dicts with gen_ai.* keys
+        # Keep them as Python objects for counting; build JSON strings below
+        trajectory_prompts = trajectory_data["trajectory_prompts"]
+        trajectory_completions = trajectory_data["trajectory_completions"]
@@
-        json_trajectory_prompts = json.dumps(trajectory_prompts)
-        json_trajectory_completions = json.dumps(trajectory_completions)
-
-        return json_trajectory_prompts, json_trajectory_completions, final_completion
+        if trajectory_prompts:
+            json_trajectory_prompts = json.dumps(trajectory_prompts)
+        else:
+            json_trajectory_prompts = json.dumps([])
+
+        if trajectory_completions:
+            json_trajectory_completions = json.dumps(trajectory_completions)
+        else:
+            json_trajectory_completions = json.dumps([])
+
+        return json_trajectory_prompts, json_trajectory_completions, final_completion

This preserves correct counts and produces "[]" in the “no data” case instead of a double-encoded string.

packages/traceloop-sdk/traceloop/sdk/experiment/utils.py (1)

90-103: Be cautious resetting TracerWrapper singleton and reinitializing Traceloop in-process

Deleting TracerWrapper.instance and calling Traceloop.init(...) unconditionally is fine for a dedicated experiment runner, but in a long‑lived process (e.g., app using Traceloop elsewhere) this will tear down and replace the existing tracer configuration, potentially surprising callers that already initialized tracing.

If this utility is intended strictly for standalone experiments, consider documenting that explicitly; otherwise you may want a code path that:

• Detects an existing initialized tracer and either reuses it, or
• Makes re‑initialization opt‑in via a parameter.

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

packages/traceloop-sdk/pyproject.toml (1)

124-124: Reconsider broad module exclusion from type checking in a strict mypy configuration.

The mypy configuration enforces disallow_untyped_defs = true (line 105), yet you're excluding an entire new module from type checking. This creates an inconsistency and introduces a potential maintenance burden. The enriched summary mentions "dynamic typing patterns," but excluding a full module is a code smell that suggests untyped code could hide runtime type errors.

Consider one of these alternatives:

1. Properly type the module to align with the strict configuration (preferred)
2. Use narrower exclusion for specific functions/classes that genuinely require dynamic typing (e.g., using # type: ignore comments on specific lines)
3. Document why broad exclusion is necessary if it must remain (e.g., in a comment or ADR)
4. Add type stubs for the dynamic parts if the code genuinely can't be typed

Can you clarify why the entire utils.py module requires exclusion from type checking? If there are specific functions with dynamic patterns that resist typing, we could use more targeted suppression instead of excluding the whole module.

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.